Skip to content

Conversation

benluddy
Copy link
Contributor

@benluddy benluddy commented Sep 25, 2025

Writes from the instance of resourcesynccontroller in the operator can contend with writes from the recovery sidecar's instance of the same controller. This is a preexisting issue, and the controller doesn't have an effective way to limit resync frequency. Adding some fixed latency to each sync in this way sets an upper bound on the churn rate when the two instances get into a feedback cycle.

The separate issues that can create a feedback cycle (e.g. the resourcesynccontroller degraded operator status condition is shared by both instances) should be separately addressed "soon".

#881

Writes from the instance of resourcesynccontroller in the operator can contend with writes from the
recovery sidecar's instance of the same controller. This is a preexisting issue, and the controller
doesn't have an effective way to limit resync frequency. Adding some fixed latency to each sync in
this way sets an upper bound on the churn rate when the two instances get into a feedback cycle.

The separate issues that can create a feedback cycle (e.g. the resourcesynccontroller degraded
operator status condition is shared by both instances) should be separately addressed "soon".
@ingvagabund
Copy link
Member

/approve

@ingvagabund
Copy link
Member

/label backport-risk-assessed

@openshift-ci openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Sep 25, 2025
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 25, 2025
@ingvagabund
Copy link
Member

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 25, 2025
Copy link
Contributor

openshift-ci bot commented Sep 25, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: benluddy, ingvagabund

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@benluddy benluddy changed the title [release-4.20] Throttle resourcesynccontroller on pairs with dual writers. [release-4.20] OCPBUGS-55217: Throttle resourcesynccontroller on pairs with dual writers. Sep 25, 2025
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Sep 25, 2025
@openshift-ci-robot
Copy link

@benluddy: This pull request references Jira Issue OCPBUGS-55217, which is invalid:

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Writes from the instance of resourcesynccontroller in the operator can contend with writes from the recovery sidecar's instance of the same controller. This is a preexisting issue, and the controller doesn't have an effective way to limit resync frequency. Adding some fixed latency to each sync in this way sets an upper bound on the churn rate when the two instances get into a feedback cycle.

The separate issues that can create a feedback cycle (e.g. the resourcesynccontroller degraded operator status condition is shared by both instances) should be separately addressed "soon".

#881

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@benluddy
Copy link
Contributor Author

/cherry-pick release-4.19

@openshift-cherrypick-robot

@benluddy: once the present PR merges, I will cherry-pick it on top of release-4.19 in a new PR and assign it to you.

In response to this:

/cherry-pick release-4.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@dgoodwin
Copy link

/jira refresh

@openshift-ci-robot
Copy link

@dgoodwin: This pull request references Jira Issue OCPBUGS-55217, which is invalid:

  • expected dependent Jira Issue OCPBUGS-62255 to be in one of the following states: MODIFIED, ON_QA, VERIFIED, but it is POST instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@dgoodwin
Copy link

/test e2e-aws-ovn-upgrade

@davegord
Copy link

/jira refresh

@openshift-ci-robot openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Sep 25, 2025
@openshift-ci-robot
Copy link

@davegord: This pull request references Jira Issue OCPBUGS-55217, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.20.0) matches configured target version for branch (4.20.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note text is set and does not match the template
  • dependent bug Jira Issue OCPBUGS-62255 is in the state MODIFIED, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
  • dependent Jira Issue OCPBUGS-62255 targets the "4.21.0" version, which is one of the valid target versions: 4.21.0
  • bug has dependents

Requesting review from QA contact:
/cc @wangke19

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot removed the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Sep 25, 2025
@openshift-ci openshift-ci bot requested a review from wangke19 September 25, 2025 22:37
@dgoodwin
Copy link

/verified bypass

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Sep 26, 2025
@openshift-ci-robot
Copy link

@dgoodwin: The verified label has been added.

In response to this:

/verified bypass

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link
Contributor

openshift-ci bot commented Sep 26, 2025

@dgoodwin: Overrode contexts on behalf of dgoodwin: ci/prow/e2e-aws-ovn-upgrade

In response to this:

/override ci/prow/e2e-aws-ovn-upgrade

https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-kube-controller-manager-operator/882/pull-ci-openshift-cluster-kube-controller-manager-operator-release-4.20-e2e-aws-ovn-upgrade/1971336188816003072 has passed, still stuck in deprovision

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-merge-bot openshift-merge-bot bot merged commit 5a7da72 into openshift:release-4.20 Sep 26, 2025
11 checks passed
@openshift-ci-robot
Copy link

@benluddy: Jira Issue OCPBUGS-55217: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-55217 has been moved to the MODIFIED state.

In response to this:

Writes from the instance of resourcesynccontroller in the operator can contend with writes from the recovery sidecar's instance of the same controller. This is a preexisting issue, and the controller doesn't have an effective way to limit resync frequency. Adding some fixed latency to each sync in this way sets an upper bound on the churn rate when the two instances get into a feedback cycle.

The separate issues that can create a feedback cycle (e.g. the resourcesynccontroller degraded operator status condition is shared by both instances) should be separately addressed "soon".

#881

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-cherrypick-robot

@benluddy: new pull request created: #884

In response to this:

/cherry-pick release-4.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-merge-robot
Copy link
Contributor

Fix included in accepted release 4.20.0-0.nightly-2025-09-27-153025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants